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Abstract 

Background: Low survey response rates in general practice are common and lead to loss of power, selection bias, 
unexpected budgetary constraints and time delays in research projects. 

Methods: Objective: To assess the effectiveness of recruitment strategies aimed at increasing survey response rates 
among GPs. 

Design: Systematic review. 

Search methods: MEDLINE (OVIDSP, 1948-2012), EMBASE (OVIDSP, 1980-2012), Evidence Based Medicine Reviews 
(OVIDSP, 2012) and references of included papers were searched. Major search terms included GPs, recruitment 
strategies, response rates, and randomised controlled trials (RCT). 

Selection criteria: Cluster RCTs, RCTs and factorial trial designs that evaluate recruitment strategies aimed at 
increasing GP survey response rates. 

Data collection and analysis: Abstracts identified by the search strategy were reviewed and relevant articles were 
retrieved. Each full-text publication was examined to determine whether it met the predetermined inclusion 
criteria. Data extraction and study quality was assessed by using predetermined checklists. 

Results: Monetary and nonmonetary incentives were more effective than no incentive with monetary incentives 
having a slightly bigger effect than nonmonetary incentives. Large incentives were more effective than small 
incentives, as were upfront monetary incentives compared to promised monetary incentives. Postal surveys were 
more effective than telephone or email surveys. One study demonstrated that sequentially mixed mode (online 
survey followed by a paper survey with a reminder) was more effective than an online survey or the combination 
of an online and paper survey sent similtaneously in the first mail out. Pre-contact with a phonecall from a peer, 
personalised packages, sending mail on Friday, and using registered mail also increased response rates in single 
studies. Pre-contact by letter or postcard almost reached statistical signficance. 

Conclusions: GP survey response rates may improve by using the following strategies: monetary and nonmonetary 
incentives, larger incentives, upfront monetary incentives, postal surveys, pre-contact with a phonecall from a peer, 
personalised packages, sending mail on Friday, and using registered mail. Mail pre-contact may also improve response 
rates and have low costs. Improved reporting and further trials, including sequential mixed mode trials and social 
media, are required to determine the effectiveness of recruitment strategies on GPs' response rates to surveys. 
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Background 

GPs play an important role in the design and implementa- 
tion of health services and health policy to improve the 
health outcomes of their patients [1]. Understanding and 
measuring GPs' knowledge, attitudes, behaviours, prac- 
tices and their views on solutions to health care issues are 
paramount to improve the quality of health care. Surveys 
are a useful tool to measure a wide variety of issues that 
are relevant to general practice. It has been argued that 
GPs are often time poor, are difficult to recruit for re- 
search studies [1,2] and often have low survey response 
rates [2] whereas the latter has been contended by others 
[3]. There has been a steady increase in research studies 
examining methods to increase response rates amongst 
doctors [3]. Despite the greater evidence base for improv- 
ing response rates, a recent review of the primary care lit- 
erature between 2000 and 2009 showed that GP response 
rates to postal surveys over the past decade are relatively 
unchanged [3]. The authors found that the average re- 
sponse rate was 61% (95% confidence interval (95% CI): 
59% to 63%) amongst 371 GP surveys. Reasons for not 
participating in research are many, including concerns 
about disrupting routine practice [1,4], time [1,2,4], rele- 
vance of the study topic [1,2], confidentiality [1,2] and 
receiving many surveys a week. The general practice 
workforce is also increasingly working part-time [5]. Low 
response rates can lead to methodological biases, such 
as non-response bias, time delays in research projects, 
budgetary problems [6], underpowered studies and raise 
an ethical dilemma in that people have been subjected to 
research that can not show its effect [7] . 

Several systematic reviews have examined strategies 
amongst the general population to increase response rates 
to postal, electronic and telephone surveys [8,9]. Whilst 
general population reviews [7,8,10] are useful, it is likely 
that some strategies are more effective in increasing GP 
survey response rates than others. For example, GPs are 
probably less sensitive to monetary incentives than the 
general population given their higher income status. Many 
of the strategies applied to the general population have 
also been developed and tested to improve GP survey 
response rates such as monetary incentives [11], question- 
naire length [12] and pre-contact [13]. Two systematic 
reviews [2,14] have examined how best to increase 
response rates amongst physicians. The first systematic 
review included studies conducted until 1999 and found 
that pre-notification, personalised mailouts and nonmone- 
tary incentives were not associated with improved physician 
response to surveys [14]. On the contrary, monetary incen- 
tives, stamps and shorter surveys led to increased response 
rates. The second systematic review included studies until 
2006 and broadly explored two intervention categories: in- 
centives and design-based approaches. The authors found 
that even small financial incentives improved physician 



response rates but that token nonmonetary incentives were 
less effective. Postal and telephone surveys were more 
effective than fax and online surveys, as were mixed mode 
surveys, first class stamps, shorter surveys, personalised 
letters, and studies endorsed by reputable professional 
organisations [2]. The aim of this study was to assess the 
effectiveness of recruitment strategies aimed at increasing 
response rates of GPs to complete surveys. 

Methods 

Criteria for considering studies for this review 
Types of studies 

Cluster randomized controlled trials, randomized con- 
trolled trials and factorial trial designs that aimed to im- 
prove survey response rates amongst GPs. 

Population 

The primary population included GPs or family physicians. 
Types of methods 

Any intervention that compares different recruitment 
methods of GPs to complete surveys. Interventions 
aimed directly at patients were excluded. Studies that 
evaluated retention strategies were excluded. Interven- 
tions that recruited GPs for clinical trials were also 
excluded. 

Types of outcome measures 

The primary outcome measure was the proportion of 
eligible GPs who responded to surveys. 

Search methods 

MEDLINE ( OVIDSP- 1948 to December 2012), EMBASE 
(OVIDSP- 1980 to December 2012), Evidence Based 
Medicine (EBM) Reviews (OvidSP - December 2012) and 
references of included papers, related literature and 
systematic reviews were searched. Major search terms 
included GPs, recruitment strategies, response rates and 
randomised controlled trials (RCT). The search strategy 
was built on previous searches [1,8,15,16]. The search 
strategy for Medline can be found in Additional file 1. 
Search results were merged into reference management 
software and duplicate records were removed. 

Data extraction 

Abstracts identified by the search strategy were reviewed 
and relevant articles were retrieved. Titles and abstracts 
were independently checked by SP. A random number 
generator was used to select about 10% of the initial 
MEDLINE electronic citations. SP and SWP independ- 
ently assessed 106 citations. Agreement of 99% was 
achieved and a Prevalence and Bias Adjusted Kappa 
Kappa of 0.98 (95% CI 0.79 to 1.17) [17]. Full-text papers 
were screened to determine eligibility by both SP and 
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SWP. Disagreements were resolved by discussion. SP and 
TV extracted data. Ambiguities were resolved by discussion. 

Assessment of risk of bias in included studies 

Two authors (SP and TV) independently assessed risk of 
bias using standard Cochrane criteria, including allocation 
concealment, random sequence generation, blinding of 
participants and personnel, complete outcome data and 
absence of reporting bias. Ambiguities were resolved by 
discussion. Intention-To-Treat analysis includes everyone 
who has been randomised to a treatment group, irrespect- 
ive of participant's non-compliance (eg not taking part in 
the treatment), protocol deviation or withdrawal from the 
study. Treated analysis is considered as high risk of 
complete outcome data bias, therefore the results of the 
assessment shows this element as a separate item for 
additional clarity. We considered absences of other bias in 
the form of whether differences in characteristics (eg age, 
gender) between study groups were reported. 

Data analysis 

For each trial we calculated results in terms of odd ratios 
and their 95% CIs. Studies that were similar in terms of 



interventions were pooled by using Review Manager 5.2 
software [18]. Heterogeneity was assessed with the I 2 test 
and Q statistics. An I 2 value greater than 50% and P < 0.1 
was indicative of heterogeneity [19]. Random effects 
models were mainly used because of significant hetero- 
geneity in some analyses. 

Results 

Figure 1 displays the flowchart of the study selection 
process [20]. A total of 1873 records were identified 
through database searches. Reference lists of related litera- 
ture and systematic reviews were also searched for add- 
itional papers [2,7,9,16,21-23]; Fourteen studies were 
identified, of which, one was a duplicate. Sixty-seven full- 
text articles were retrieved to determine eligibility. Forty- 
three studies were excluded with the following reasons: no GP 
data (n = 31), not an RCT (n = 6), meta-analyses or literature 
review (n = 3), clinical trial (n = 3) and two articles could not 
be retrieved. A total of 23 studies were included in this review. 

We divided the strategies in two broad categories based 
on previous work conducted in this area: incentives and 
design based strategies [2]. There were 15 RCTs, four two- 
by-two factorial randomized design trials, one three-arm, 



Records identified through 
database searching and duplicates 
removed 
(n = 1873) 



Additional records identified 
through other sources 
(n = 13) 



Records screened 
(n =1886) 



Figure 1 Flowchart. 



Full-text articles assessed 
for eligibility 
(n = 67) 



Studies included in 
qualitative synthesis 
(n = 23) 



Studies included in 
quantitative synthesis 
(meta-analysis) 
(n = 23) 



Full-text articles excluded, 
with reasons 
(n = 1819) 



Full-text articles excluded, 
with reasons 
(n = 43) 

• NoGP data (n=31) 

• Not an RCT(n=6) 

• Meta-analysis or 
literature review (n=3) 

• Clinical trial (n=3) 
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two four-arm and one five-arm parallel randomized 
design trials (Table 1). There were three Australian, three 
American, one British and one Canadian study that 
measured the effectiveness of incentives. Nine Australian, 
three American, two Canadian, one Danish, and one Irish 
study included design based strategies. The majority of 
survey topics focussed on patient care, clinical guidelines 
and two studies [24,25] focussed on GPs personal work 
situation (eg job satisfaction). A summary of the risk of 
bias in included studies is presented in Table 2. 

Incentives 
Monetary incentives 

Four studies examined the effects of monetary incentives 
on GPs' response to surveys. Deehan and colleagues [11] 
found that cash payment and charity donation (£5 and 
£10) were more effective in increasing GP response rate 
compared to no incentive (pooled OR 1.87, 95% CI 1.48 
to 2.36) (Table 3). The study also demonstrated that cash 
incentives were more effective than charity donations 
compared to no incentive, and larger cash incentives 
were more effective than smaller cash incentives. 

Asch and colleagues [26] also found that a larger cash 
incentive (US$5) was more effective than a smaller cash 
incentive (US$2) (OR 1.83, 95% CI 1.4 to 2.35). Everett 
and co-workers found that a $1 bill was more effective 
than no incentive. James and colleagues [27] found that 
upfront payment of a cash incentive (US$25) was signifi- 
cantly more effective than a promised cash incentive 
(US $25) (OR 2.88, 95% CI 1.70 to 4.89). 

The extent of the risk of bias in all four studies examin- 
ing monetary incentives is unclear as there was insuffi- 
cient detail to determine if allocation of subjects were 
concealed, if participants and personnel were blinded to 
the study. In two studies it was not clear if there was ab- 
sence of other bias. Missing data due to loss of follow up 
(eg undelivered surveys) was equally distributed between 
study groups in two studies [26,28], but this information 
was not reported in the other two studies [11,27]. Only 
James and co-workers [27] did not undertake an intention 
to treat analysis, which may have over estimated the effect 
of the intervention. 

Nonmonetary incentives 

Four studies [25,29-31] examined the effects of nonmone- 
tary incentives compared to no incentives on GP response 
rate to surveys. The weighted overall effect size showed a 
small but significant association between nonmonetary in- 
centives and GP response (OR 1.39, 95% CI 1.15 to 1.69). 
Thomson et al. [32] compared two types of lottery incen- 
tives, and found that a lottery for one chance to win a large 
prize (6 bottles of champagne) was more effective than six 
chances to win one bottle (OR 1.47, 95% CI 1.05 to 2.08). 
All five studies that assessed nonmonetary incentives were 



randomised control trials, four undertook intention to treat 
analysis and all reported on pre-determined primary out- 
comes. The adequacy of allocation concealment and blind- 
ing of participants and personnel are unclear in four 
studies. Four studies did not report missing data due to 
loss to follow up, whilst only Robertson et al. [31] reported 
that missing data were equally distributed between study 
groups. No significant differences in demographic charac- 
teristics between study groups were found in Baron et al. 
[29] and Pit et al. [25], whilst this risk of bias was not clear 
in Robertson et al. [31], Thomson et al. [32] and Mclaren 
and Shelley [30]. 

Size of incentives 

Three studies[ll,26,32] found that a larger incentive had 
a small but significant effect on response rates (OR 1.47, 
95% CI 1.19 to 1.81). 

Finally, Ward and co-workers [33] used a pen as an in- 
centive and compared it with other recruitment methods 
but did not find that a pen increased the response rates. 
They did find that women were more likely to respond 
to a pen. 

Design based strategies 

One study [34] found that asking general questions first 
do not significantly increase GP survey response rates. 

Three studies [35-37] demonstrated that postal surveys 
are significantly more effective than telephone or email 
surveys (OR 1.82, 95% CI 1.19 to 2.78). Scott and col- 
leagues [24] found that an online survey did not lead to in- 
creased response rates compared to using mixed methods. 
However, they found that sending a letter containing login 
details and an option to request a paper copy followed by a 
reminder that included login details and a paper copy led 
to increased response rates when compared to online 
surveys. 

Two studies [34,38] found that pre-contact with GPs 
via a postcard or letter compared to no intervention 
increased the response rate but this was just not statisti- 
cally significant (OR 1.16, 95% CI 0.99 to 1.37). 

In terms of mode of the pre-contact, Gattellari et al. 
[13] found that pre-contact by fax was not statistically 
different to pre-contact by mail in increasing GP response 
rates. Gupta et al. [39] found that there was no statistical 
difference between pre-contact by a medical researcher 
compared to a non-medical researcher. But a study con- 
ducted by Ward and colleagues [33] demonstrated that an 
upfront phonecall from a peer led to increased response 
rates when compared to 3 other methods (pen, letter and 
research assistant prompt). 

Bonevski and co-workers [40] found that reminder tele- 
phone calls to non-responders prior to the 3 rd mailout did 
not significantly increase response rates when compared 
to not conducting reminder telephone calls. Akl and co- 



Table 1 Details of included studies 



Study ID 



Aims of 
study 



Study 
sample 



Study 
topic 



Study 
design 



Total 

number of 
subjects in 
study (N) 



Control (n) 



Intervention (n) 



Country 



3 5 



^ 3- 

00 5- 
po £ 

^ 2 



Akl et al., Assess the effect of 1) tracking 

2011 [41] responses and 2) day of mailing 

(Monday vs Friday) on physician 
survey response rate 



Asch et al., To evaluate the effect of 
1998 [26] incentive size by providing 

physicians either a $5 bill 

or a $2 bill. 

Baron et al., To evaluate the cost 

2001 [29] effectiveness of a lottery on 

physicians response rates to a 

mail survey. 

Bonevskiet al, To examine the efficacy of two 
201 1 [40] strategies for improving genera 

practitioner response to a survey 
and a secondary aim was to 
assess GPs self reported 
preferred mode of survey 
administration. 

Deehan et al, To explore the effects of 
1997 [11] financial and non-financial in- 

ducements on response rates to 
increase the overall response 
rate. 



Directors of 
Family Medicine 
residency 
programs 
(ie GPs) 

Primary care physicians 
identified from the 
American Medical 
Association Physician 
Master File 

Family Doctors 
identified from 
membership list of the 
Quebec Federation of 
GPs 

GPs practising in NSW 
selected from the 
Australasian Medical 
Publishing Company 
database. 



A random sample of all 
CP's in England and 
Wales. 



Primary care physicians 
working in Ireland 



Drummond To investigate the effect of two 

et al, 2008 [34] low cost interventions (pre- 

contact and questionnaire order) identified from various 
on response to a primary care sources, 
physician postal survey 

Everett et al. To examine the effectiveness Family Physicians 
1997 [28] of $1 incentive on response rates 

Gattellari et al, Assess effectiveness of two GPs known to be in 

2012 [13] response-aiding strategies in a current practice 

postal survey 



Training of 
residents in 
implementation 
of clinical 
guidelines 

Attitudes about cost 
containment in cancer 
screening. 



The determinants of 
influenza immunization 
among family 
physicians in Quebec. 

The knowledge, 
attitudes and practices 
of GPs around vitamin 
D. 



Survey on GP's work 
and attitudes with 
alcohol misusing 
patients. 



A national survey 
assessing views and 
practices of physicians 
regarding prostate 
specific antigen (PSA) 
testing. 

Firearm-safety 
counseling beliefs 

Management of 
nonvalvular atrial 
fibrillation 



Two by two 
factorial 
randomized 
design 

RCT 



RCT 



RCT 



Five arm 
parallel trial 
design with 
randomization 



Two by two 
factorial 
randomized 
design 



456 



1000 



1000 



334 (Cover 
letter trial), 
1166 

(Telephone 

reminder 

trial) 

3584 



RCT 

Two by two 
factorial 
randomized 
control trial 



1599 



600 



1000 



No tracking 
n = 228, 
Monday 
mailing 
n = 228 

$2 cash incentive 
received in 
initial mailing 
n = 500 



No lottery n = 500 



Standard cover letter 
N = 167, No 
telephone 
reminder n = 576 



No inducement 



No pre-contact 
n = 743, Version 
2 survey (Topic 
specific questions 
first) n = 744 

No incentive n=300 

Mail prompt n = 500, 
Coloured seal without 
text n = 500 



Tracking n = 228, USA 
Friday mailing 
n = 228 



$5 cash incentive USA 
received in initial 
mailing n = 500 



Lottery n = 500 Canada 



Standard letter Australia 

plus local 

division of 

general practice 

cover letter 

n = 167, Telephone 

reminder n = 590 

£5 charity donation UK 
n = 607, £10 charity 
donation n = 578, 
£5cash payment 
n = 613, £10 cash 
payment n = 598, 
(Total for all 
interventions n = 2396) 

Pre-contact Ireland 
n = 715, Version 
1 survey (general 
questions first) 
n = 714 

$1 bill n=300 USA 

Fax prompt Australia 
n = 500, 

Coloured seal + 
text n = 500 



-a 

a; 

ID 
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Gupta et al. To determine the effectiveness 
1 997 [39] of a telephone prompt by a 

medical researcher compared to 
a nonmedical research assistant 
in improving survey response 
rate of GP's, and to compare 
personnel costs. 

Hocking et al., To compare GP response to a 
2006 [35] telephone interview with 

response to a postal survey with 

three reminders. 



James et al., To study the effects of payment 
201 1 [27] timing, form of payment and 

requiring a social security 
number (SSN) on survey 
response rates. 



A national random 
sample of Australian 
GPs 



A random sample of 
Victorian GPs 



Practising US physicians 
ages 65 and under 
representing all 
specialties (including 
primary care), selected 
from the AMA Physician 
Masterfile 



McLaren and To compare effect of a primer Victorian GPs 
Shelley 2000 telephone call and postcard on 
[30] GP response rate; and to 

compare the effect of informing 

GPs about a prize for 

participating in survey compared 

to not informing GPs of the 

prize. 

Maheux et al. To determine the response rates Quebec Physicians 
1989 [42] of Quebec physicians by sending 

a handwritten thank you letter 

and by sending a more 

personalized mailout package. 

Olivarius and To test a possible day the week GPs 
Andreasen, effect on doctors response rate 
1995 [43] to a postal questionnaire. 



A national survey 
assessing GPs views of 
clinical practice 
guidelines 



Survey to assess 
knowledge and 
diagnostic and 
management practices 
of genital chlamydia 
infection. 

Survey on ethical and 
moral beliefs. 



RGT 



404 



Management issues 
surrounding early 
pregnancy bleeding 
and miscarriage 



To determine the level 
of physicians support 
for a number of patient 
care issues. 

A nationwide survey on 
doctors' attitude on the 
relative importance of 
general practice and 
the specialties in the 
treatment of general 
diseases and diseases 
commonly regarded as 
belonging to the 
specialties. 



RCT 



Four arm 
parallel 
design with 
randomization 



Two by two 
factorial 
design with 
randomization 



443 



621 



RCT 



RCT 



Second 



mailing 
N = 356, 



Telephone 
prompt by a 
medical researcher 
n = 202 



Postal survey 
n = 451 



Promised $US25 
check requiring 
SSN n = 102 



Telephone prompt 
by experienced 
nonmedical research 
assistant n = 202 



Telephone interview 
n = 416 



Immediate $US25 cash 
n = 129, 
Immediate 
$US25 check 
n = 97, Promised 
$US25 check not 
requiring SSN 
n = 115 



Australia 



Australia 



USA 



Telephone n=305 Postcard n=316 

No prize notification N=310 Prize notification n=31 1 



Australia 



No handwritten 
postscript 
n = 186, Non- 
Third mailing personalised 
N = 253 mailout n = 127 



200 



Dispatch of survey 
on Thursday (for 
receipt before 
weekend) n = 98 



Handwritten 
postscript 

n = 170, Personalised 
mailout n = 126 

Dispatch of survey 
on Saturday (for 
receipt after weekend) 
n = 102 



Canada 



Denmark 



3 a 



i 2 

^ 3- 

i o 

oo g- 

^ 2 

a* 4^ 



-a 
a; 
IO 

n> 
o 
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Pedrana et al, 
2008 [44] 



Pirotta et al, 
1 999 [38] 



Pit et al, 
2013 [25] 



Robertson 

et al, 2005 [31] 



Scott et al. 
201 1 [24] 



To compare response rates to 
postal surveys sent by norma 
post and registered mail and to 
assess the cost implications of 
the two mailing methods. 

To measure the effect of a 
primer postcard to improve 
mailed survey response rates. 

To assess the effectiveness of 
small non-conditional non- 
financial incentive (ie attractive 
pen) on survey response rates 

To test the effect of a $AUD2 
scratch lottery ticket on response 
rates 



To compare effects and costs of 
three different modes of survey 
administration in a national 
survey of Doctors 



General practitioners in 
Victoria. 



Seguin et al, 
2004 [37] 



To compare email with regular 
mail for conducting surveys of 
family physicians. 



Shosteck and Comparison of mail and 
Fairweather, personal administration of 
1979 [36] questionnaire among physicians. 



To assess antenatal 
screening practices in 
Australia 



Victorian GPs who had General Practitioners 
at least 1500 attitudes to 

consultations in 1995-96 complementary 
therapies. 



RCT 



RCT 



Practicing GPs in 
Northern Rivers region 
of NSW 



A random sample GP's 
and specialists from the 
medical directory of 
Australia. 

A stratified random 
sample of doctors 
undertaking clinical 
practice from a national 
directory of all doctors 
in Australia. 
Stratification was by 
doctor type and rural/ 
remote category. 



How to improve RCT 
sustainable employment 
of ageing GPs in rural 
Australia 



To explore the uptake 
of 1 9 new drugs into 
the clinical practices of 
Australian GPs and 
specialists. 

Study of the dynamics 
of the medical labour 
market in Australia, 
focusing on workforce 
participation and its 
determinants among 
Australian doctors. 



A random sample of Survey on use of 
physicians listed in the sildenafil citrate (Viagra) 
college of family 
physicians of Canada's 
database who had an 
email address 

Office-based primary Physician antibiotic 



care physicians who 
were in active practice 
within the States of 
Washington DC and 
Maryland, and treating 
upper respiratory or 
lower urinary tract 
infections selected from 
the AMA database. 



prescription practices. 



RCT 



1550 



soo 



125 



464 



3-arm parallel 
trial design 

with 

randomisation 



1091 



RCT 



2397 



RCT 



543 



Normal mai 
n = 775 



No primer postcard 
n = 400 



Registered mail 
n = 775 



Australia 



Primer postcard Australia 
n = 400 



No pen incentive n = 62 Pen incentive n = 63 Australia 



No lottery n = 232 



Online survey, followed 
by reminder letter with 
login details n = 369 



Survey delivered 
by post to the 
physicians 
without email 
n = 800 



Mail technique 
n = 296 



AUD$2 scratch lottery Australia 
n = 232 



Sequential mixed Australia 

mode -online survey, 

followed by reminder 

letter with paper 

survey n = 388, 

Simultaneous mixed 

mode -paper survey 

and login details sent 

together, with 

reminder letter with 

login details n = 334 

Survey delivered by Canada 
email n = 798, Survey 
delivered by post to 
the physicians with 
e mail n = 799 

Personal interview USA 
technique n = 247 



Table 1 Details of included studies (Continued) 



Thomson et al. 


To maximise the response to a 


Practicing GP's selected 


GP's attitudes to the 


RCT 568 Lottery of one chance to 


Lottery of six chances UK 


2004 [32] 


postal questionnaire and to test 


from the Lothian 


management of 


win six bottles of 


to win one bottle of 




the most effective form of 


Primary Care NHS Trust 


ingrowing toenails. 


champagne n = 286 


champagne n = 282 




incentive. 


database. 








Ward et al. 


To evaluate response aiding 


Randomly selected 


Cancer screening 


4-arm parallel 1550 


Doctor advance phone Australia 


1 998 [33] 


strategies feasible in large 


sample of general 




trial design 


prompt: n = 249, Pen 




surveys 


practitioners 




with 


prompt: n = 261 , Letter 










randomisation 


prompt: n = 260, Non- 












doctor advance phone 












prompt: n = 501 



Pit et al. BMC Medical Research Methodology 2014, 14:76 
http://www.biomedcentral.com/1471-2288/14/76 



Page 9 of 14 



Table 2 Quality assessment of included studies 



Study ID 



Allocation Randomisation Blinding of 
concealed? performed? participants & 

personnel? 



Complete Absence of Absence of ITT 
outcome data? reporting bias? other bias? analysis? 



Monetary incentives 

1. Asch et al., 1998 [26] ? 

2. Deehan et al.,1997 [11] ? 

3. Everett etal.,1 997 [28] ? 

4. James et al, 201 1 [27] ? 
Non-monetary incentives 

5. Baron et al., 2001 [29] ? 

6. McLaren and Shelley, ? 
2000 [30] 

7. Pit et al., 2013 [25] X 

8. Robertson et al., ? 
2005 [31] 

9. Thomson et al, ? 
2004 [32] 

Questionnaire design/ mode of delivery 

10. Drummond et al, V 
2008 [34] 

1 1. Hocking et al, X 
2006 [35] 

12. Shosteck and ? 
Fairweather,1979 [36] 



13,Seguin et al.,2004 [37] ? 

14. Scott et al., 201 1 [24] X 
Other design based strategies 

15. Akl et al, 2011 [41] X 

16. Bonevski et al, ? 
2011 [40] 

17. Gattellari et al, X 
2012[13] 

18. Gupta et al, 1997 [39] ? 

19. Maheux et al, ? 
1 989[42] 

20. Olivarius and X 
Andreasen, 1995 [43] 

21. Pedrana et al, X 
2008 [44] 

22. Pirotta etal.,1 999 [38] V 

23. Ward et al, 1998 [33] X 

Legend: 
? — > Unclear. 
X^No. 
V^Yes. 



workers [41] found that tracking of responses did not help 
to improve response rates over not tracking responses. 
(Table 4) 

Personalisation of mail- out packages to GPs generally 
does not appear to increase response rates. Inclusion of 
a professional sponsorship letter, coloured seal with text 



and hand written thank you postscripts did not increase 
survey response rates [13,40,42]. However, Maheux et al. 
[42] found that a personalised mail-out package that in- 
cluded the physician's title, name and address individu- 
ally typed onto the envelope, hand stamped outgoing 
envelopes identified to the university and hand stamped 
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return envelope increased response rates with modest 
statistical significance (OR 1.73, 95% CI 1.04 to 2.87). 

Aid and colleagues [41] found that Friday morning 
was more effective than a Monday mailing. However, the 
pooled results of Aid et al. [41] and Olivarius and 
Andreasen [43] showed that timing of mailing at the end 
of the week compared to during the week does not effect 
survey response rates amongst GPs. Pedrana and co- 
workers [44] found that surveys sent by registered post 
significantly increased the response rates compared to 
surveys sent by standard post (Table 4). 

The majority of included studies that assessed the im- 
pact of design based strategies reduced the risk of bias 
by randomisation, reporting of predetermined primary 
outcomes and intention to treat analysis. A high risk of 
selection bias due to inadequate allocation concealment 
was evident in half (7 of 14) of these studies. Blinding of 
participants and personnel was undertaken in only one 
study, not undertaken in two of the studies and unclear 
in 11 studies. 

Missing data were equally distributed between study 
groups in seven studies and unclear in the remaining 
seven studies. The balance of demographic characteris- 
tics between study subjects was demonstrated in five 
studies and unclear in the other eight studies (Table 2). 



Discussion 

This review identified several strategies that can easily be 
implemented by researchers and policy makers to increase 
response rates for GP surveys. Monetary and nonmonetary 
incentives were more effective than no incentive. Large in- 
centives were more effective than small incentives as were 
upfront monetary incentives compared to promised mon- 
etary incentives. Postal surveys were more effective than 
telephone or email surveys. One study demonstrated that 
sequentially mixed mode (online survey followed by a 
paper survey with the reminder) was more effective than 
an online survey or the combination of an online and 
paper survey sent similtaneously in the first mail-out. Pre- 
contact with a phonecall from a peer colleague, persona- 
lised packages, sending mail on Friday, and using registered 
mail also increased response rates in single studies. Pre- 
contact with a letter or postcard increased reponse rates 
slightly but just did not reach statistical significance. 

This review focussed specifically on GPs to assist in in- 
creasing GP survey response which limits its generalisabil- 
ity to other doctor groups. There is evidence that GPs and 
specialists respond differently to recruitment strategies. 
For example, Robertson and colleagues found that incen- 
tives had a quantitatively larger effect amongst GPs com- 
pared to specialists response rates but this was not 



Table 3 Monetary and non-monetary incentives 


Monetary incentives 


Intervention 


OR 


95% CI 


Deehan et al, 1997 [11] 


£10 vs no incentive 


3.14 


2.37, 4.15 




£5 vs no incentive 


2.22 


1 .66, 2.98 




£10 charity donation vs no incentive 


1.20 


0.85, 1.69 




£5 charity donation vs no incentive 


1.12 


0.79, 1.57 




Pooled (Deehan, 1997) 


1.87 


1.48, 2.36 


Everett et al., 1997 [28] 


$1 bill vs no incentive 


1.86 


1 .34, 2.57 




Pooled 


1.87 


1.55, 2.26 


Nonmonetary incentives 








Baron et al., 2001 [29] 


Lottery to win weekend trip vs No incentive 


1.31 


1 .02, 1 .70 


McLaren and Shelley, 2000 [30] 


Awareness of prize vs no awareness of prize 


1.17 


0.84, 1.61 


Pit et al, 2013 [25] 


Pen vs no incentive 


1.85 


0.91, 3.77 


Robertson et al, 2005 [31] 


AU$2 Lottery vs no incentive 


1.78 


1 .23, 2.60 




Pooled 


1.39 


1.15, 1.69 


Large vs small incentives 








Asch et al, 1 998 [26] 


US$5 vs US$2 


1.83 


1 .43, 2.35 


Deehan et al, 1997 [11] 


£10 cash vs £5 cash 


1.41 


1.06,1.87 




£10 charity donation vs $5 


1.08 


0.73,1.58 


Thomson et al, 2004 [32] 


Lottery to win a large prize (6 bottles of champagne) 
vs six chances to win one bottle 


1.47 


1 .05, 2.08 




Pooled 


1.47 


1.19,1.81 


Prepaid vs promised incentives 








James et al, 201 1 [27] 


US$25 prepaid vs US$25 promised 


2.88 


1 .70, 4.89 
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Table 4 Design based strategies 



Questionnaire design 


Intervention 


OR 


95% CI 


Drummond et al., 2008 [34] 


General questions vs topic related questions first 


1.17 


0.96, 


1.43 


Survey delivery mode 


Intervention 








Hocking et al, 2006 [35] 


Postal survey vs telephone interview 


2.66 


2.02, 


3.52 


Shosteck and Fairweather, 1 979 [36] 


Postal survey vs telephone interview 


1.03 


0.68, 


1.55 


Seguin et al, 2004 [37] 


Postal survey vs email survey 


2.00 


1.67, 


2.38 




Pooled 


1.82 


1.19, 


2.78 


Scott et al. 201 1 [24] 


Online survey vs mixed mode 


0.73 


0.51, 


1.04 




Online survey vs sequential mixed mode 


0.65 


0.44, 


0.97 




Online survey vs Simultenous mixed mode 


0.83 


0.55, 


1.26 


Advance contact 










Drummond et al, 2008 [34] 


Letter pre-contact letter vs no pre-contact 


1.10 


0.91, 


1.35 


Pirotta et al, 1999 [38] 


Postcard pre-contact vs no pre-contact 


1.30 


0.98, 


1.72 




Pooled 


1.16 


0.99, 


1.37 


Gattellari et al., 2012 [13] 


Mail pre-contact vs fax pre-contact 


1.11 


0.87, 


1.43 


McLaren and Shelley, 2000 [30] 


Telephone pre-contact vs postcard pre-contact 


1.01 


0.73, 


1.40 


Gupta et al., 1997 [39] 


Medical researcher pre-contact vs non-medical researcher 


1.33 


0.87, 


2.05 


Ward et al, 1998 [33] 


GP phonecall pre-contact vs other interventions 


1.39 


1.02, 


1.88 




Pen pre-contact vs other interventions 


0.97 


0.73, 


1.30 




Letter pre-contact vs other interventions 


0.95 


0.71, 


1.26 




Research assistant phonecall pre-contact vs other interventions 


0.86 


0.68, 


1.09 


Reminders 










Bonevski et al, 201 1 [40] 


Telephone reminder vs none 


1 11 


0.68, 


2.15 


Tracking 










Akl et al, 2011 [41] 


Tracking of reponses for follow up vs No tracking 


0.85 


0.59, 


1.23 


Personalisation and GP group endorsed letters 








Bonevski et al, 201 1 [40] 


Sponsorship letter vs standard letter 1 st mailout 


1.38 


0.86, 


2.22 


Gattellari et al, 2012 [13] 


Coloured seal with text vs coloured seal without text 


1.08 


0.84, 


1.38 


Maheux et al, 1989[42] 


Handwritten 'thank you' postscript with 1st reminder 
vs no postscript 


1.45 


0.92, 


2.30 


Maheux et al, 1989[42] 


Personalised mail out package vs Non-personalised mail 
out package with 2 nd reminder 


1 73 


1.04, 


2.87 


Timing of mailing 










Akl et al, 2011 [41] 


Friday vs onday mailing 


1.86 


1.28, 


2.69 


Olivarius and Andreasen, 1995 [43] 


Saturday vs Thursday mailing 


0.55 


0.25, 


1.23 




Pooled 


1.07 


0.33, 3.50 


Type of mailing 










Pedrana et al, 2008 [44] 


Registered mail vs standard mail 


2.97 


2.31, 


3.81 



statistically significant [31]. Scott and co-workers [24] 
found that GPs had the lowest response rates and special- 
ists the highest response rates. Specialists had statistically 
significant higher response rates compared to other doc- 
tors when offered an online and paper survey at the same 
time. On the contrary, Olivarius and Andreasen [43] con- 
cluded that GPs had higher response rates than specialists. 
Furthermore, Maheux and colleagues [42] reported that 



specialists were a little more sensitive to the interventions 
than the GPs. Lasdy, James and colleagues found that 
there was no difference between doctor types and the im- 
pact of incentive type and timing on survey response rates 
[27]. The difference in reponse rates per type of doctor 
may well depend on a variety of factors including socio- 
demographic factors, the topic of interest and the level of 
involvement required to participate in the survey. This 
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should be taken into account when designing recruitment 
strategies for surveys. 

We found substantial heterogeneity among the included 
studies which makes it difficult to generalise the study 
findings and determine the impact of other factors on the 
study results [3]. For example, in the general population 
more sensitive topics lead to lower survey response rates 
[8]. This was difficult to assess in this systematic review 
because there were not any study topics that were consid- 
ered to be sensitive for GPs. In the study published by 
Seguin and colleagues in 2004 [37], the difference in 
response rate between postal and email surveys could be 
attributed to the year that the study was conducted 
because GPs may now be more used to email. A more re- 
cent study [24] found that online surveys led to lower re- 
sponse rates than mixed mode approaches (both online 
and paper based surveys). The reluctance of GPs to 
complete surveys online is confirmed in other studies 
[40] . Potential explanations are concerns about confidenti- 
ality, familiarity and access to the internet, and the ease of 
completing the survey online ( eg the need to recall or 
enter a password). It is possible that online response rates 
may improve in the future because of the growing use of 
smartphones and portable devices that make it easier and 
quicker to access online surveys. The number of re- 
minders also may have had an impact on study results. 
The largest impact of incentives and other recruitment 
strategies is often seen in the first wave of recruitment and 
this steadily decreases with followup reminders [25,26]. 

Researchers must take into account the available re- 
sources when designing survey recruitment strategies. 
Shorter surveys, online surveys, the number of re- 
minders and smaller incentives can all lead to cost- 
savings but this needs to be carefully balanced with max- 
imising high quality, valid and reliable data. Eleven stud- 
ies in this systematic review included information on 
intervention costs [24,26,29-31,34,36,38,39,41,44]. Three 
out of four studies which used nonmonetary incentives 
provided relevant cost information. Baron et al. [29], 
Mclaren and Shelley [30] and Robertson et al. [31] found 
that by offering a nonmonetary incentive at an additional 
cost of CAN$16, AU$3.46 and AU$23.42 per survey 
returned led to an increased response rate of respectively 
6%, 4% and 15%. Asch and co-workers [26] found that a 
$5 incentive versus a $2 incentive cost only an additional 
$0.53 cents per survey returned with an increase in 
response rates of 15%. 

Amongst design based studies, Scott and colleagues 
[24] found that the additional cost per 1% increase in 
response rate was AU$3290 for sequential mixed mode 
strategies and AU$10,156 for simultaneous mixed mode 
strategies when compared to the online mode. The 
additional cost per additional response was AU$6.07 for 
sequential mixed mode strategies and AU$18.75 for 



simultaneous mixed mode strategies with an increase in 
GP response rates of 6% and 2% respectively when com- 
pared to the online survey (response rate 14%). The 
sequential mixed mode strategy therefore appears to be the 
most cost-effective method. However, these cost figures 
need to be weighed up against the fact that there was some 
response bias in the sequential and simultenous mixed 
mode strategies and a higher rate of item non-response in 
the online survey. Researchers should be aware that there 
is some evidence that there are new differences between 
physician responders and non-responders and early and 
late responders, suggesting that there is a low level of non- 
response bias [2,14]. It is argued that the reason for this is 
that the population is rather homogeneous. 

Little differences in cost per respondent were found in 
studies that used a letter or postcard as a pre-contact 
strategy. Drummond and co-workers [34] reported that 
sending a postcard led to an additional cost of €0.87 per 
respondent and an increase in the reponse rate of 4%. 
The non-postcard group had a response rate of 46% 
compared to 50% in the postcard group. Similarly, 
Pirotta et al. [38] found that sending a postcard added 
only AU$0.40 per respondent but increased the reponse 
by 6% from 60% to 66%. McLaren and co-workers [30] 
found no difference between telephone pre-contact and 
postcard pre-contact but that the costs involved in call- 
ing GPs was about 5 times as much as sending a post- 
card. Gupta and colleagues [39] found that pre-contact 
by a medical practitioner and a research assistant led to 
the similar costs because the research assistant had to 
make more calls and there was no statistical significant 
difference in response rates. It therefore appears that 
pre-contacting GPs by mail is a cheap method to im- 
prove response rates compared to doing nothing, and is 
cheaper than telephone pre-contact which may lead to 
similar response rates. Lastly, Pedrana and co-workers 
[44] found that sending surveys by registered mail cost 
an additional AU$1531.50 but led to a 19% increase in 
response rates (response rate: 86% registered mail versus 
67% normal mail.). This seems to suggest that extra 
costs are justified in these cases. 

In accordance with our study, systematic review studies 
amongst physicians have also found that monetary incen- 
tives increase response rates [2,14], as does pre-payment 
[2,14]. Contrary to our findings, Van Geest and Kellerman 
found that non-monetary incentives did not increase 
response rates amongst physicians [2,14]. This may also 
reflect differences between GPs and other physicians. 
VanGeest et al. [2] pointed out that non-monetary in- 
centives are more likely to work if physicians value them. 
Large incentives were more effective than small incentives 
in our study and Kellerman and Herold's study [14]. How- 
ever, Vangeest found that larger incentives led to mixed 
results for both monetary and non-monetary incentives [2]. 
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Kellerman and Herold [14] found no difference between 
mail and telephone interviews, whereas our review did. 
This may be explained by the fact that our review included 
more recent studies. VanGeest also reported that family 
physicians prefer mail surveys [2]. The general consensus 
is that GPs still prefer mail surveys over online surveys. 
Pre-contact with a phonecall from a peer colleague, perso- 
nalised packages, sending mail on Friday, and using regis- 
tered mail also increased response rates in single studies. 
Kellerman and Herold [14] found that pre-notification 
was not effective but this conclusion was based on only 
one study. Kellerman and colleagues reported one study 
that found that personalised mailouts were effective dur- 
ing the 1 st mail out but no effect was found during 2 nd 
mailouts in two other studies. However, Maheux et al. 
[42] demonstrated an effect of personalised mailouts dur- 
ing 3 rd mailouts. This effect is likely to be explained by the 
fact that the researchers included multiple elements to 
personalise the package. 

The quality of the included studies generally was not 
clear from current reporting. Allocation concealment and 
blinding of participants were clearly reported in only two 
studies. Complete outcome data were available for 10 
studies. Absence of reporting bias was present in all stud- 
ies, whereas absence of other bias was only applicable to 9 
studies. The strength of this review is that , to our know- 
ledge, this is the first systematic review that has examined 
GPs as a group and not a sample of multiple different 
medical practitioners. Another strength of this review is 
that the outcome measure is an objective outcome meas- 
ure and is therefore less likely to be influenced by report- 
ing bias within studies. A weakness of the study was that 
we did not approach authors for additional information 
due to the budget constraints. This has an impact on judg- 
ing the quality assessment of the included studies as men- 
tioned above, in particular in the area of blinding of 
participants and personel. Finally, a limitation of the study 
is that the majority of studies were conducted in English 
speaking countries and may therefore limit generalsibility 
to other countries. 

Further research is required to advance this field of re- 
search. Areas that need further exploring include strat- 
egies and factors such as survey topic, confidentiality 
guarantees, level of incentives, use of social media, mobile 
phone applications, and sequential mixed method applica- 
tions. Furthermore, better reporting is required to deter- 
mine the quality of the included studies. We support 
Creavin and colleagues' suggestion to develop a standard 
template for survey studies, similar to the consort state- 
ment [3] to improve reporting on survey methodology. 
Finally, we also recommend that future studies measure 
and report on the resources used when conducting the 
interventions to guide future researchers in strategy selec- 
tion. We recommend that primary care researchers can 



build randomised controlled trials into survey research to 
further test which strategies are most effective. 

Conclusions 

GPs response rates to surveys may improve by using the 
following strategies: monetary and nonmonetary incen- 
tives, larger incentives, upfront monetary incentives, postal 
surveys, pre-contact with a phonecall from a peer col- 
league, personalised packages, sending mail on Friday, and 
using registered mail. Mail pre-contact may also improve 
response rates and has low costs. Improved reporting and 
further trials, including mixed mode studies, are required 
to determine the effectiveness of recruitment strategies on 
general practitioners' response rates to surveys. 
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